
1 METHOD AND APPARATUS FOR ACCESSING ELECTRONIC 

2 DATA VI A A FAMILIAR PRINTED M EDIUM 

3 CRO SS REFERENCE TO RELATED APPLICATIONS 

4 This application is a continuation-in-part of application 

5 serial number 08/628,246, filed April 4, 1996. 

6 FIELD OF THE INVENTION 

7 The present invention relates generally to the electronic 
s*%8 media industry -- such as cable television (CATV) , home shopping 
v«j9 services, on-line computer services, personal computer 

#o applications, and CD-ROM-based multi -media applications -- and, 

;; Fii more particularly, to a method and apparatus for allowing a user 

;^i2 to access and make use of such electronic media services via a 

?Hi3 device that makes use of a standard book, magazine or other 

14 printed work. 

15 

16 BACKGROUND OF THE I NVE NTION 

n It is a well-known fact that a large fraction of the 

18 population is unfamiliar with the operation or use of modern 

19 computer devices. This remains true despite significant advances 

20 in user- interface technology such as mouses, windows, menus 

21 and the like. It is commonly said that such technology makes 

22 computers "user friendly." However, the modern 

-1- 



mouse/window/menu- based user- interface technology has been widely 
available for several years, and yet it still appears that this 
technology is not a panacea for computer-phobia. Indeed; studies 
have shown that a majority of VCR owners cannot operate the 
simple menu-based interface used to program their VCRs. Thus, 
there exists a great need for improved user-interface technology, 
if computer-based electronic media is to become as widely used 
and accepted as television or radio. 

This need for improved user- interface technology will become 
even greater with the arrival of technology such as the presently 
planned "information superhighway," which will permit delivery of 
high-bandwidth (i.e., full-motion video rate) digital data into 
millions of homes via fiber optic, cable, RF, microwave or 
satellite links. Such technology will, in concept, permit 
instant delivery of a virtually limitless selection of 
commercial, informational, educational and entertainment 
programming at a user's request. However, without better user- 
interface technology, the average user may not be capable of 
enjoying the vast capabilities of such a system. Indeed, it is 
clear that the number of programming choices available to the 
user of such technology will be far greater than the number of 
choices involved in programming a VCR --a task already 



1 demonstrated to be too complicated for the average user of 

2 present-day user- interface technology. 

3 In contrast to the difficulty many people encounter in using 

4 remote controls and other prior art computer interfaces, printed 
s matter -- such as books and magazines -- represents an almost 

6 universally familiar and non- intimidating medium by which a user 

7 can acquire desired information. Even illiterate individuals 

8 incapable of reading text can nonetheless peruse pages of printed 
M matter and appreciate the substance of flashy advertisements and 

15 the like. Thus, it would be highly desirable to provide a system 
lit with the information accessing capabilities of a modern CD-ROM or 
~H on-line computer system, and the user- interface simplicity of 

Jff printed matter. 

M The prior art includes a class of devices known as "talking 

it! books" -- see, e.g., U.S. Patent Nos . 4,636,881 entitled TALKING 

16 BOOK WITH AN INFRARED DETECTOR USED TO DETECT PAGE TURNING, 

17 4,702,573 entitled VISUAL AND AUDIBLE ACTIVATED WORK AND METHOD 

18 OF FORMING SAME, 4,778,391 entitled SOUND- PRODUCING AMUSEMENT OR 

19 EDUCATIONAL DEVICES, 4,809,246 entitled SOUND ILLUSTRATED BOOK 

20 HAVING PAGE INDICATOR CIRCUIT, 4,990,092 entitled TALKING BOOK 

21 and 5,2 09,665 entitled INTERACTIVE AUDIO VISUAL WORK, all of 

22 which are incorporated herein by reference. Typically, these 
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1 "talking books" consist of a book with various sensors which 

2 when activated by touching, page turning, etc. -- cause a sound 

3 generating means (also embedded within the book) to produce or 

4 replay particular sounds. Talking books thus provide an 

5 interface for allowing an unsophisticated user (i.e., a child) to 

6 access a very primitive computer (i.e., the sound generating 

7 means embedded within the book) via familiar printed matter 

8 (i.e., the book with embedded sensors). Importantly, however, 

ifj talking books do not provide a means for interfacing with modern 

m electronic media -- such as cable television (CATV) , home 

hh shopping services, on-line computer services, CD-ROM-based multi- 

Ilk media applications, interactive TV or home computer applications. 

One approach to interfacing with /these modern electronic 

s& media is the "simulated book" -- see, e.g., U.S. Patent No. 

jj$ 4,855,725, entitled MICROPROCESSOR BASED SIMULATED BOOK, 

16 incorporated herein by reference. The "simulated book" is in 

17 essence a book-size intelligent graphics terminal. Unlike the 

is talking books, the simulated book is not a self-contained system, 

19 but rather transmits commands to and receives data from a CD-ROM 

20 equipped personal computer via a wireless link. Thus, the 

21 programming that the simulated book can access is not limited to 

22 that which can be stored in embedded memory devices, as with the 

-4- 



I 1 



1 talking books. Importantly, however, the user- interface provided 

2 by the simulated book is essentially the conventional computer 

3 interface i.e., keys, pointer, menus, etc. Therefore, a 

4 computer-phobic user will likely still find the simulated book 

5 intimidating and inaccessible. 

6 A disadvantage of both the talking book and simulated book 

7 technologies is that both include relatively costly electronics - 

8 - i.e., microprocessors, memory, display devices, etc. as a 
,g part of the "book." Thus, these technologies cannot be 

;ip effectively used to create a "throw-away" interactive magazine, 

In newspaper or advertising brochure. 

Thus, there remains a need for a method and apparatus for 

■B accessing the vast resources of electronic media using a device 

as familiar and non- intimidating as printed matter. There 

jjig remains a further need for such a method and apparatus which 

16 utilizes a low cost, throw-away printed matter. 

17 

18 SUMMARY OF THE INVENTION 

19 One object of the present invention is a method and 

20 apparatus for allowing a user to access electronic media via a 

21 printed matter. 

22 Another object of the invention is a method and apparatus 
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1 for allowing a user to access electronic media relating to, or 

2 expanding upon, material presented in the printed matter. 

3 Another object of the invention is a low cost, throw-away 

4 printed matter useful in connection with other objects of the 

5 invention. 

6 Still another object of the invention is an improved method 

7 of providing electronic media services. 

8 Yet another object of the invention is an intelligent 
# controller for use in connection with the invention. 

In accordance with one embodiment, the invention comprises: 

^2 (i) a printed matter having at least one sensor and a transmitter 

■f] associated therewith; and (ii) an intelligent controller having a 

S receiver and a means for accessing programming material. A user 

m triggers said sensor through interaction with said printed 

\is matter, for example, by touching a particular spot on a page or 

16 by turning a page. In response to the triggering of said sensor, 

17 the transmitter sends a signal indicative of said sensor. The 

18 receiver receives said signal and, in response thereto, the 

19 intelligent controller executes a pre-programmed command related 

20 to accessing or controlling electronic media or programming. For 

21 example, when the user triggers a sensor associated with an 

22 advertisement in the printed matter, the intelligent controller 
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may, in response, send a signal via a telephone line, cable 
connection, or wireless modem or cellular link to a remote video 
server, and thereby cause a promotional program to appear on the 
user 1 s television . 

In accordance with another embodiment, the invention 
comprises: (i) a printed matter having at least one machine- 
recognizable feature such as a bar code or magnetic strip (or 
any commonly used printed indicia, such as a printed character, 
symbol or pictorial icon) , (ii) a feature recognition unit having 
a means for recognizing said feature and a transmitter, and (iii) 
an intelligent controller having a receiver. The user directs 
the feature recognition unit to a feature on said printed matter. 
In response, said recognition unit transmits a signal indicative 
of the identity of the particular feature. The receiver receives 
said signal and the intelligent controller, in response thereto, 
executes an appropriate pre-programmed command. 

In accordance with another embodiment, the invention 
comprises: (i) a printed matter having at least one machine- 
recognizable feature and (ii) an intelligent feature recognition 
unit having means for recognizing said feature, means for 
associating said recognized feature with a command, and means for 
issuing said command over a wireless link. In use, the user 
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1 directs said intelligent feature recognition unit at a feature on 

2 said printed matter. In response, said intelligent recognition 

3 unit associates said feature with a pre-programmed command and 

4 issues a pre-programmed command sequence over a wireless data 

s link to control or access electronic media services. The command 

6 may, for example, be transmitted -- via an infrared (IR) or 

7 ultrasound link -- to a CATV control box in the same room, or -- 

8 via a cellular or satellite link --to the CATV company office. 
J In accordance with another embodiment, the invention 

i| comprises a printed matter having: (i) at least one sensor, (ii) 

'll a control module and (iii) a transmitter associated therewith. 

"ii In response to the triggering of said sensor, said control module 

Jj| directs the transmitter to transmit a command related to 

m accessing or controlling an electronic media service. 
\$s Other aspects of the invention relate to methods of 

16 providing, accessing or utilizing electronic media services. In 

n accordance with one such aspect, the invention involves: (i) 

is providing a printed matter having at least one sensor associated 

19 therewith, (ii) providing an intelligent controller which, in 

20 response to the triggering of said sensor, performs a pre- 

21 programmed command, and (iii) executing said pre-programmed 

22 command to access or control an electronic media. 
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Another aspect of the invention involves a method of 
providing electronic media services, which includes the steps of: 
(i) providing printed matter to a potential customer and (ii) 
pre -programming an intelligent controller to access or control an 
electronic media service in response to an event wherein the 
customer interacts with the printed matter in a particular 
manner. Advantageously, said printed matter comprises a low 
cost, throw- away publication. 

In accordance with another aspect of the invention, an 
improved method of providing shop-at-home services includes the 
steps of: (i) providing to the customer a printed catalogue 
having at least one sensor or machine -recognizable feature 
associated therewith, (ii) programming a controller to execute a 
pre-programmed command in response to an event wherein the 
customer interacts with said sensor or feature, and (iii) 
providing a service -- e.g., displaying promotional programming 
on the customer's television, contacting the customer by 
telephone, establishing a computer "chat" link, etc. --by 
telephone, cable, or wireless link in response to the execution 
of said command. 

Another aspect of the present invention relates to an 
improved method of instruction, including the steps of: (i) 
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1 providing an instructional printed matter -- such as a textbook, 

2 cookbook, children's book or manual -- having at least one sensor 

3 or machine -recognizable feature associated therewith, (ii) 

4 providing a means, distinct from said textbook or other printed 

s matter, for executing a pre-programmed command in response to an 

6 event wherein a reader interacts with said sensor or feature, and 

7 (iii) in response to said command, causing or controlling: (a) 

JaK 8 the electronic delivery or presentation of information related to 

■^9 that in the textbook or other printed matter; and/or (b) the 

,'fio establishment of a communication link to a live tutor or 

a gu consultant familiar with the subject matter contained in the 

s 12 instructional printed matter. 

^13 Another aspect of the invention relates to a low cost, 

f^i4 throw-away printed matter including at least one machine- 

'^is recognizable feature adapted for use in connection with the 

16 invention. 

17 

18 BRIEF DESCRIPTION OF THE DRAWINGS 

19 The above --as well as other aspects, objects and 

20 features of the present invention will be described in the 

21 Detailed Description below, which is intended to be read in 

22 conjunction with the following set of drawings, in which: 
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t Fig. 1 depicts an embodiment wherein the display unit 

2 is embedded within the printed matter; 

3 Fig. 2 depicts an embodiment of the invention wherein 

4 electronic media is presented on a user's TV set; 
s Fig. 3 depicts an embodiment of the invention wherein 

6 programming material is accessed from a remote 

7 source; 

8 Fig. 4 depicts an embodiment of the invention which 

9 includes a feature recognition unit; 

10 Fig. 4a depicts an embodiment of the invention wherein 
n the feature recognition unit provides an interface 

12 between the display unit and a remote source of 

13 programming material; 

14 Fig. 5 depicts an embodiment of the invention adapted 
is for presentation of musical programming; 

16 Fig. 5a depicts an alternative embodiment of the 

17 invention adapted for presentation of musical 
is programming; 

19 Fig. 6 depicts an embodiment of the invention wherein 

20 the display unit comprises a personal computer; 

21 Fig. 6a depicts an embodiment of the invention wherein 

22 the user employs a hand-held scanner/pointer 
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1 device to select features associated with a 

2 printed matter and to interface with an 

3 intelligent controller or personal computer; 

4 Fig. 6b depicts an alternative embodiment of the 

5 invention wherein the user employs a separately 

6 attached trackball mouse and hand-held scanner 

7 device to select features associated with a 

8 printed matter and to interface with an 

9 intelligent controller or personal computer; 

to Fig. 6c depicts another embodiment of the invention 

n wherein the user employs a separately attached 

12 trackball mouse and digital camera device to 

13 select features associated with a printed matter 

14 and to interface with an intelligent controller or 
is personal computer; 

16 Fig. 6d depicts another embodiment of the invention 

17 wherein the user employs a microphone set up to 

18 interface with an intelligent controller or 

19 personal computer which contains voice recognition 

20 software to select features associated with a 

21 printed matter; 

22 Fig. 6e depicts yet another embodiment of the invention 
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wherein the user employs a traditional keyboard 
set up to interface with an intelligent controller 
or personal computer for manual entry to select 
features associated with a printed matter; 

FIG. 6f depicts another embodiment of the present 

invention in which a scanable magnetic strip is 
set up to interface with an intelligent controller 
or personal computer equipped with a magnetic card 
reader to select features associated with a 
printed matter; and 

Fig. 7 depicts an embodiment of the invention adapted 
for shop-at-home applications. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In this section, the various preferred embodiments of the 
invention are described from two general perspectives. The 
first, a "functional" perspective, focuses on the contemplated 
interactions between the user and the various components -- i.e., 
the printed matter, controller, display unit, etc. -- of the 
invention. This functional description provides the insight 
needed to implement the software or firmware used in connection 
with the invention. The second perspective, the "apparatus" 
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1 view,, describes the various technologies that can be used to 

2 implement the individual components of the present invention. 

3 

4 THE FUNCTIONAL PERSPECTIVE 

s Reference is now made to Fig. 1, which depicts an 

6 embodiment of the invention comprising a printed matter 1 in 

7 communication (preferably wireless) with a data server 2. 

\% Printed matter 1 includes at least one sensor 3 and a controller, 

"'"'4 which preferably includes a microprocessor 4. A stiff or flexible 

!io page 5 (any page within a book) holds a display screen 6. 

5 a E Printed matter 1 can take the form of a book, magazine, 

m manual, musical score, catalog, advertisement, newspaper, 

•!i3 telephone or electronic service directory, or other like means. 

-34 The controller — including microprocessor 4 is preferably 

is embedded within the spine or any other page of printed matter 1. 

16 Display screen 6 can be an LED display, a passive or active 

n matrix LCD display or other like means, and may also have an 

is audio transducer associated therewith. 

19 Sensor 3 is preferably touch sensitive, but can also be a 

20 page sensor or a combination of touch and page sensor, as 

21 described below. In response to a user's touch, microprocessor 4 

22 causes programming material to be retrieved from data server 2 
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and displayed on screen 6 - Data communication between server 2 
and microprocessor 4 may operate via RF cellular, microwave, IR, 
optical, conductive, telephonic or CATV links, or any combination 
of these or other like means. 

Reference is now made to Fig. 2, which depicts an 
alternative embodiment of the invention comprising a printed 
matter 21, an intelligent controller 24 and display unit 25. 
Printed matter 21 includes at least one sensor 22 and a 
transmitter 23. In response to a user's actuation of sensor 22 , 
transmitter 23 transmits a coded signal indicative of the 
identity of the actuated sensor. A receiver 26 in intelligent 
controller 24 receives the coded signal. Controller 24 then 
identifies the actuated sensor, and initiates display of 
appropriate programming material on display unit 25 (which can be 
a TV set or any other means for audio or audiovisual 
presentation, including but not limited to a personal computer) . 
Programming material preferably derives from a mass storage 
device e.g., a magnetic disk, CD-ROM, ROM, flash RAM, PCMCIA 
card or other memory means associated with intelligent 

controller 24 (or with display unit 25) • As used herein, the 
term "memory means" shall also include future storage 
technologies, such as the recently announced multi- layer CD-ROMs 



1 being developed by IBM. See "New I.B.M. Laser Method Stacks Data 

2 on Disks," New York Times , May 13, 1994. 

3 Reference is now made to Fig. 3, which depicts an embodiment 

4 of the invention in which intelligent controller 31 derives 

5 programming material from a remote server 30. Controller 31 

6 includes means for accessing a remote server 30 of programming 

7 material, and preferably further includes means for decompressing 

8 compressed programming material received from server 30. In 

9 response to a coded signal received by receiver 26, intelligent 

10 controller 31 sends an appropriate command to server 30 to select 

11 and initiate transfer of appropriate programming material. 

12 Controller 31 then receives programming material from remote 

13 server 30 and prepares (e.g., decompresses, if necessary) the 

14 material for presentation on display unit 25. As with 

15 conventional pay-per-view CATV services and on-line computer 

16 services, the user is billed according to the volume and/or 

17 nature of programming material requested. 

18 Reference is now made to Fig. 4, which depicts an embodiment 

19 of the invention including a printed matter 40, a feature 

20 recognition unit 42, an intelligent controller 31, a server 3 0 

21 and a display unit 25, Printed matter 4 0 includes at least one 

22 machine-recognizable feature 41 in the printed work, such as a 
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1 being developed by IBM. See "New I.B.M. Laser Method Stacks Data 

2 on Disks," Tvtew York Tittirr, May 13, 1994. 

3 Reference is now made to Fig. 3, which depicts an embodiment 

4 of the invention in which intelligent controller 31 derives 

s programming material from a remote server 30. Controller 31 

6 includes means for accessing a remote server 3 0 of programming 

7 material, and preferably further includes means for decompressing 

8 compressed programming material received from server 30. In 

9 response to a coded signal received by receiver 26, intelligent 

10 controller 31 sends an appropriate command to server 30 to select 
n and initiate transfer of appropriate programming material. 

12 Controller 31 then receives programming material from remote 

n server 30 and prepares (e.g., decompresses, if necessary) the 

14 material for presentation on display unit 25. As with 

is conventional pay-per-view CATV services and on-line computer 

16 services, the user is billed according to the volume and/or 

17 nature of programming material requested. 

18 Reference is now made to Fig. 4, which depicts an embodiment 

19 of the invention including a printed matter 40, a feature 

20 recognition unit 42, an intelligent controller 31, a server 30 

21 and a display unit 25. Printed matter 40 includes at least one 

22 machine-recognizable feature 41 in the printed work, such as a 
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1 view, describes the various technologies that can be used to 

2 implement the individual components of the present invention. 

3 

4 THE FUNCTIONAL PERSPECTIVE 

5 Reference is now made to Fig. 1, which depicts an 

6 embodiment of the invention comprising a printed matter 1 in 

7 communication (preferably wireless) with a data server 2. 

r| Printed matter 1 includes at least one sensor 3 and a controller, 

4 which preferably includes a microprocessor 4. A stiff or flexible 

% page 5 (any page within a book) holds a display screen 6. 
■|3 Printed matter 1 can take the form of a book, magazine, 

rn manual, musical score, catalog, advertisement, newspaper, 

Fto telephone or electronic service directory, or other like means. 

?3"4 The controller -- including microprocessor 4 is preferably 

is embedded within the spine or any other page of printed matter 1. 

16 Display screen 6 can be an LED display, a passive or active 

17 matrix LCD display or other like means, and may also have an 

18 audio transducer associated therewith. 

19 Sensor 3 is preferably touch sensitive, but can also be a 

20 page sensor or a combination of touch and page sensor, as 

21 described below. In response to a user's touch, microprocessor 4 

22 causes programming material to be retrieved from data server 2 
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,and displayed on screen 6. Data communication between server 2 

2 and microprocessor 4 may operate via RF cellular, microwave, IR, 

3 optical, conductive, telephonic or CATV links, or any combination 

4 of these or other like means. 

5 Reference is now made to Fig. 2, which depicts an 

6 alternative embodiment of the invention comprising a printed 

7 matter 21, an intelligent controller 24 and display unit 25. 
Printed matter 21 includes at least one sensor 22 and a 
transmitter 23. In response to a user's actuation of sensor 22, 

HO transmitter 23 transmits a coded signal indicative of the 

, li identity of the actuated sensor. A receiver 26 in intelligent 

■ f f 

rf 12 controller 24 receives the coded signal. Controller 24 then 

13 identifies the actuated sensor, and initiates display of 

14 appropriate programming material on display unit 25 (which can be 
is a TV set or any other means for audio or audiovisual 

16 presentation, including but not limited to a personal computer) . 

n Programming material preferably derives from a mass storage 

is device -- e.g., a magnetic disk, CD-ROM, ROM, flash RAM, PCMCIA 

19 card or other memory means associated with intelligent 

20 controller 24 (or with display unit 25) . As used herein, the 

21 term "memory means" shall also include future storage 

22 technologies, such as the recently announced multi-layer CD-ROMs 
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"bar code, invisible bar code, magnetic code, printed character, 
symbol" or pictorial icon, or other feature. 

Feature recognition unit 42 is a hand-held device and 
includes means 44 for recognizing feature 41, a transmitter 45 
and an optional user actuation switch 43. The user directs 
recognition unit 42 at a feature and depresses actuation switch 
43. In response thereto, means 44 for recognizing "reads" (i.e., 
for a printed feature, scans and identifies) the feature 41, and 
transmitter 45 transmits a coded signal indicative of the 
identity of feature 41. The operation of controller 31, server 
30 and display unit 25 is otherwise identical or similar to the 
Fig . 3 embodiment . 

Still referring to Fig. 4, use of embedded features 41 as 
opposed to sensors lowers the fabrication cost of printed 
matter 40. Advantageously, printed matter 40 can be a low cost, 
throw-away publication. 

An identification code generated either by recognition unit 
42 or intelligent controller 31 allows the user to be billed for 
his/her use of transmitted material by server 3 0 and, for 
commercial applications, allows the advertiser to identify the 
potential customer . 

Reference is now made to Fig. 4a, which depicts an 
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1 embodiment of the invention in which the functions of the 

2 intelligent controller are integrated into a recognition/control 

3 unit 46. Unit 46 includes means 44 for recognizing feature 41 

4 and means 4 8 for accessing — preferably via a cellular RF link - 
s - programming material associated with server 30. In addition, 

6 recognition/control unit 46 optionally includes a microprocessor. 

7 Either recognition/control unit 46 or display unit 47 may include 
Js j means for decompressing compressed programming material. 

Cp Reference is now made to Fig. 5, which depicts an embodiment 

igo of the invention adapted for musical applications. The 

s pi embodiment includes a printed musical score 50, an electronic 

-= 12 baton 51, a music controller 55 and an audio/audiovisual player 

^tt 56. Using baton 51, the user directs a recognition means 52 

]!}u which can be either a CCD camera or a magnetic detector at a 

1 * is particular portion 57 of printed score 50. Transmitter 53 then 

16 transmits a coded signal indicative of the selected portion 57. 

n Music controller 55, in response to the coded signal, directs the 

is presentation of appropriate audio or audiovisual programming on 

19 player 56. Music controller 55 may include a means for storing 

20 programming material such as ROM, CD-ROM, flash RAM, PCMCIA 

21 card or other memory means or an electronic musical 

22 synthesizer, or both. 
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1 Reference is now made to Fig. 5a, which depicts an 

2 alternative embodiment of the invention directed to musical 

3 applications. In Fig. 5a, music controller 55a retrieves 

4 programming material from a remote data server 57. Using a 

5 remote server permits continuous updating of programming 

6 material, such as replacing one performance or opera company with 

7 another. Either player 56 or controller 55a preferably includes 
13 means for decompressing compressed programming material received 
'y from server 57. 

% Reference is now made to Fig. 6, which depicts an embodiment 

]$\ of the invention utilizing a personal computer to access and 

i=}2 display electronic programming material. In this embodiment, 

HJ3 portions of the personal computer provide the functions of the 

Oi4 previously-described intelligent controller, while other portions 

is of the computer provide the functions of the display unit. The 

u scope of this invention should not be limited to the traditional 

17 notions of Personal Computers. The present invention will find 

18 use in conjunction with more non- traditional applications such as 

19 WEB-TV®, Video Phones, as well as other chip based technologies. 

20 An interface unit 60 includes a receiver 26 for receiving a coded 

21 signal from a transmitter 23, representative of an actuated 

22 sensor 22 (or, in an alternative embodiment which includes a 
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1 feature recognition unit, from the feature recognition unit 

2 indicative of the selected feature) on printed matter 21. 

3 Interface unit 60 connects to personal computer 61 by any 

4 conventional means, such as an I/O port, card slot, etc. 

5 Personal computer 61 monitors the interface unit 60, and displays 

6 (or controls the display of) appropriate programming material 

7 selected by the user through his/her interaction with printed 
fri matter 21. 

%l Programming material can be stored on personal computer 61 - 

ffi - on a ROM, CD-ROM, flash RAM, PCMCIA card, or other disk/card 

it supplied along with printed matter 21 or accessed from a 

\i remote data server 62 . 

& Reference is now made to Fig. 6a, which shows an embodiment 

M of the invention wherein a user employs a hand-held recognition 

15 device 67 to: (i) select programming material associated with a 

16 feature 41 on a printed matter 40; and (ii) control or interact 
n with a personal computer 65 during the presentation and/or 

is execution of the programming material. The recognition device 67 

19 may be comprised of one of many possible devices already in use 

20 within the computer and electronics industry. 

21 The prefered embodiment may utilize a scanner/mouse which is 

22 described in detail in U.S. Patent No. 4,804,949, entitled HAND- 
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1 HAND-HELD OPTICAL SCANNER AND COMPUTER MOUSE, which is 

2 incorporated herein by reference. The Fig. 6a embodiment 

3 provides at least two modes of interaction for the use of the 

4 Scanner/Mouse : 

5 (i) Scan Mode: 

6 In this mode, scanner/mouse 67 operates as a 

7 scanner. The user points scanner /mouse 67 at a 

$ feature 41 in a printed matter 40 and depresses a 

4] switch to cause the feature to be scanned. 

m Personal computer 65 receives the scanned image, 

iE decodes it, and executes a pre-programmed command 

12 in response. Execution of the pre-programmed 

g command may, for example, cause personal computer 

% 65 to fetch an interactive software program from a 

5 is centralized data bank 62, and begin execution of 

16 the fetched software, Once the fetched software 

17 begins executing, scanner/mouse 67 may be placed 
is into mouse /trackball mode, in order to support 

19 interaction between the user and personal computer 

20 65 . 

21 (ii) Mouse/Trackball Mode: 

22 In this mode, scanner/mouse 67 operates as a 
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normal X-Y movement sensor, such as a mouse or 
trackball. Thus, this mode is useful to control 
conventional graphical interface functions, such 
as menus, windows, icons, cursors, video games, 
etc. Those skilled in the art will appreciate 
that this mode can provide any or all features 
typically found in modern graphical user 
interfaces . 

The scanning function of the mouse may in the alternative be 
accomplished by the utilization of a bar code or other symbol 
laser scanning device incorporated within the hand-held unit 67. 
This invention is commonly used in grocery stores, and automated 
warehousing applications to price, and inventory goods. These 
scanners may be manufactured as a single unit with the trackball 
mouse of the PC 65 or alternatively may be employed as a separate 
device 69 with separately attached trackball mouse 64 as shown in 
FIG. 6b. As described in greater detail in U.S. Patent No. 
4,387,297, entitled PORTABLE LASER SCANNING SYSTEM AND SCANNING 
METHODS, these laser scanning devices 6 9 have a trigger within 
the device which initiates a repetitive scanning of each object 
bearing a symbol 41, and an indicator displays when the reading 
of that particular object has been completed. The recognition of 
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the symbol 41 is accomplished by the generation of a miniature 
light beam from the unit which is reflected in varying intensity 
depending on the qualities of the symbol 41. A sensing means 
within the unit detects the intensity of the light reflected from 
the symbol 41. The detecting means then generates an electrical 
signal indicative of the detected intensity of the light. A 
signal processing means then interprets the electrical signals to 
generate data descriptive of the symbol. The printed matter 40 
would be affixed with bar codes or other symbols 41 which could 
be read with the laser scanning device 69. The user would be 
directed to place the scanning portion of the device upon the 
symbol 41 and depress the trigger at appropriate times while 
reading or viewing the printed matter 40. The symbol 41 once 
scanned and processed would deliver a corresponding digital 
command, or reference to the software of the PC 65 working in 
coordination with the scanning device 69. 

An additional peripheral device which may be employed to 
recognize symbols could be the digital camera 63 shown in FIG. 
6c. The digital camera 63 has gained popularity over recent 
years, and would have the advantage in this application of 
already being available to consumers who have either purchased it 
for its picture quality digital images, or for those who would be 
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interested in the cameras multi-functional use including this 
application. The digital camera 63 employs solid state sensing 
devices such as a charged coupled device to record images. The 
optical images which are recorded are then converted to discreet 
electric signals, which are then stored in various memory 
devices. The images may then be recalled from the memory device 
to be displayed, printed, or otherwise manipulated. One of the 
applications of the digital camera 63 is in the downloading of 
digital images to personal computers 65. The digital camera 63 
could therefore be used to download images of symbols 41 within 
reading materials 40 to the user's PC 65. Symbol processing 
software on the PC 65 could then interpret symbols downloaded 
from the digital camera 63 or its memory, and the PC 65 program's 
code logic would thereby trigger corresponding functions of PC 
65. 

The triggering of the functions of PC 65 could also be 
initialized by verbal commands 58a from the user 58, as shown in 
FIG. 6d. The PC 65 could be affixed with a microphone 59. As the 
user 58 reads or works with a printed material 40 she could be 
instructed to pronounce verbal commands 58a into the microphone 
59 by feature 41 in the printed matter 40. The PC 65 would then 
have voice recognition software which would interpret the verbal 

-24- 



1 commands 58a from the printed matter 40 to access the programing 

2 material by, for example running applications which correspond 

3 with the commands . 

4 In yet another alternative embodiment of the invention the 

5 feature 41 of printed matter 40 could also be manually entered 

6 into the PC 65 by way of a traditional keyboard 68 shown in FIG, 

7 6e. Users of the printed matter 40 who, for example, are reading 

8 advertisements, or working through a tutorial would be directed 

9 to manually input codes or word commands (feature 41) into the PC 

10 65 by way of a keyboard 68. The command would then, similar to 

n other sensing devices, initialize a program or PC function which 

12 would operate in concert with the printed matter 40. 

13 In each of the configurations of this invention described in 

14 FIGs. 6a-6f, PC applications may be retrieved from PC interfaces 

15 62 such as CD ROM drives, hard drives, PCMCIA cards, Flash RAM, 

16 web interfaces, or any other addressable memory device which the 
n PC has access too. 

is The coded information might also be transferred to the 

19 computer through a scanable magnetic strip 41a as shown in figure 

20 6f . Known magnetic strip technology, used such as that credit and 

21 debit cards, has information as fixed electronic data embedded on 

22 the surface of the card. The PC 65 will have a magnetic card 

-25- 



"fip' 'll' 



1 reader 49 connected to one of its external ports. The printed 

2 matter 4 0 will direct the user to read the magnetic strip 41a 

3 into the reader 49. The information embedded on the magnetic 

4 strip will provide commands to the PC 65 to access programing 

5 material by directing currently running applications, or to 
e retrieve and run applications from remote or local sources. 

7 Data collected from the sensing device (s) in the form of 

8 digital images or translated to a decoded command may by 
transferred to the PC 65 via a METHOD AND SYSTEM OF Bi- 
ll 10 DIRECTIONAL PARALLEL PORT DATA TRANSFER BETWEEN DATA PROCESSING 
ifl n SYSTEMS as described in detail in U.S. Patent No. 5, 600,793. This 
O 12 process transfers discreet packets of data both to and from 

;H 13 peripheral devices. These devices would be in constant 

14 communication, allowing both the sensing device as well as the PC 

is 65 to send and receive data. Alternatively the data may be 

16 transferred via infra red, microwave or some other light 

n frequency data transfer method. The user might also desire that 

18 the sensing device be manufactured in kind with the traditional 

19 television remote control device. This would re-enforce the 

20 desirability of the unit being manufactured in a wireless unit 

21 with a light emitting data transfer method as described above. 

22 This advantage would of course be balanced with the cost 
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effectiveness of the design, as well as the reliability of the 
data transfer via either method. 

Reference is now made to Fig. 7, which depicts an embodiment 
of the invention adapted for commercial and shop-at-home 
applications. The customer views and interacts with a printed 
matter 70 (via sensors or a feature recognition units, both as 
previously described) . Printed matter 70 is preferably a throw- 
away catalog or advertising brochure listing commercial items, 
such as programming choices, merchandise, travel or event 
schedules, or television and/or radio programming schedules. The 
user interacts with printed matter 70 (i.e., with certain 
features and/or sensors therein) to cause a coded signal 71 
indicative of the customer's selection to be transmitted (either 
by a transmitter embedded in the printed matter or within a 
feature recognition unit) to controller 73. In response thereto, 
controller 73 utilizes a modem/ telephone (or other wired or 
wireless communication) link 74-75 to communicate the customer's 
selection to a commercial provider's remote office 78 (or to a 
centralized data bank or information superhighway) . 

The commercial provider preferably uses a "caller ID" 
function to identify the customer and can respond in a number of 
ways: (i) have a representative contact the customer; (ii) send 
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1 the ordered merchandise or tickets to the customer; (iii) direct 

2 a data server 72 to provide programming or additional promotional 

3 material via a CATV line 76 to controller 73, which replays the 

4 material on the customer's TV set; or (iv) download and execute 

5 an interactive merchandise selection program on the customer's 

6 personal computer or any other electronic media input, output or 

7 sensory stimulating device. 

8 

; 9 THE APPARATUS PERSPECTIVE : 

io The various technologies used to implement the individual 

n functional components that comprise a part of this invention are 

12 described below. 

13 

14 TOUCH PAD TECHNOLOGY 

is Sensor 3 can be implemented using a wide variety of 

16 presently available touch sensitive pad technology. See, for 

n example, U.S. Patent Nos. 5,016,008 entitled DEVICE FOR 

18 DETECTING THE POSITION OF A CONTROL MEMBER ON A TOUCH -SENSITIVE 

19 PAD, 5,012,124 entitled TOUCH SENSITIVE CONTROL PANEL, 4,862,151 

20 entitled REMOTE CONTROL DEVICE FOR A COMPUTER ASSOCIATED WITH A 

21 VIDEO SCREEN, 4,827,084 entitled SOLID STATE IMAGE DETECTOR AND 

22 SIGNAL GENERATOR and 4,644,101 entitled PRESSURE-RESPONSIVE 
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POSITION SENSOR, all of which are incorporated herein by 
reference . 

Referring now to Fig. 2, a plurality of touch sensors 22 are 
embedded within the back binding of the printed matter 21. 
Alternatively, touch sensors 22 can be positioned along inner or 
outer margins of the binding, or of particular pages, of printed 
matter 21 {or along an edge of a mounting tray in which a 
disposable printed matter is placed) . Electronics mounted within 
printed matter 21 (or within a mounting tray) responds to the 
actuation of a touch sensor 22 and transmits a coded signal 
indicative of the identity of the actuated sensor. 

Advantageously, touch sensors can be combined with page 
sensors as described in U.S. Patent Nos . 5,209,665; 4,990,092; 
4,636,881 and 4,809,246, all previously incorporated herein --to 
provide a larger number of "effective sensors." In this 
embodiment, each time a sensor is actuated, the electronics 
within the printed matter also checks the page sensor (s) to 
determine which page is currently being viewed. The electronics 
then generates a coded signal that identifies both the sensor 
actuated and the currently visible page. Thus, each (page, 
sensor) combination represents, in essence, a single "effective 
sensor. " 
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1 Hybrid embodiments combining touch pads and page sensors are 

2 also possible. For example, a printed matter might contain a 

3 sheet of touch sensors every so many pages, with the page 

4 interval being dependent on the stiffness of the pages and the 

5 responsiveness of the sensors. That is, if the maximum number of 

6 pages through which a user's touch could be reliably detected by 

7 a touch sensor was X, then the printed matter would preferably 

8 include a sheet of touch sensors every X pages of text. 

9 

io INTELLIGENT CONTROLLERS AND DATA/ COMMUNICATION S ERVERS 

n In accordance with several embodiments of the invention, an 

12 intelligent controller controls the selection of programming in a 

13 video "file server" system. File servers are well known in the 

14 art. Generally, as used herein, the term intelligent controller 
is can refer to computer equipment having either: (a) some type of 

16 mass storage device, typically a magnetic or magneto- optical 

17 disk, that is connected to a network and utilized as central 

is storage for multiple users via the network; and/or (b) some type 

19 of network interface which allows establishment of a 

20 communication link with other user(s) . 

21 Referring to a "file server" type of intelligent controller, 

22 the information that is typically stored on such a system 
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consists of binary computer data such as executable programs and 
corresponding data. Exemplary of the types of corresponding data 
stored are numbers for spreadsheets, ASCII characters and codes 
for word processors, and drawing commands for drawing and CAD 
programs. These or similar types of data do not generally have a 
time element associated with them, nor do they generally require 
any extra processing beyond the usual interpretation that is 
accomplished by the computer program with which they are intended 
to be used. 

In contrast, the playback of audio/video information has a 
time element associated with its use. Furthermore, the amount of 
data required to store audio/video program information in "raw" 
digital form is often prohibitive. Thus, it is common to employ 
audio/video compression techniques to compress audio/video 
program information before it is stored. As a result , further 
processing in the form of decompression and digital-to-analog 
conversion is required before the real-time viewing of an 
audio/video program can take place. Decompression may be 
accomplished through the use of known hardware or software 
techniques. Digital-to-analog conversion may be required, 
depending on the type of equipment used for viewing the 
audio/video program. A known audio/video file server apparatus 
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manufactured and marketed by Protocomm of Trevose, Pa. permits 
real-time playback of compressed audio/video program 
information. 

Generally, modern audio/video file server systems include: a 
storage unit for centrally storing compressed digital audio/video 
program information, a transceiver for receiving compressed 
digital audio/ video program information from an external source 
over a network (or wireless communication link) to update the 
stored audio/video program information, a plurality of playback 
units, each associated with an external playback line and an 
external playback station, each including a decompression unit, 
for receiving selected compressed digital audio/video program 
information from the storage unit, for decompressing the selected 
compressed digital audio/video program information received from 
the storage unit, and for playing it in real time over the 
associated playback line to the associated playback station, a 
network interface unit for receiving playback requests from the 
playback units, and a processor for controlling the storage and 
playback units to play the decompressed selected audio/video 
program information in real time for viewing by users at selected 
playback stations . 

Such a system is described, for example, in U.S. Patent No, 
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5,262,875 entitled AUDIO/VIDEO PILE SERVER INCLUDING 

DECOMPRESSION/ PLAYBACK MEANS, which is incorporated herein by 

reference. The '875 system can be adapted to operate in 

accordance with the present invention by coupling intelligent 

controller of the present invention with the '875 system's 

network interface unit (which provides the playback requests to 

the playback unit). In the '875 system: 

"Playback may be initiated through a user 
request from one of the playback stations or 
through direct interaction with audio/video 
file server 5. In the latter case, a simple 
user interface, such as a menu interface, 
may be employed for selection of the desired 
program information and the playback 
station. For example, a menu of the available 
items of audio/video program information, as 
named by the user or by the system, may be 
presented. The user may make a selection 
through cursor control keys on a keyboard." 

In contrast, the present invention permits the user to select and 

control the presentation of audio/video programming through a 

familiar printed matter interface, as has been previously 

described. The present invention thus offers the advantage of 

nearly universal accessibility. 

In accordance with the present invention, multimedia 

programming is preferably delivered from the file server (s) to 

the replay unit via an Integrated Service Digital Network (ISDN) . 
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1 U.S. Patent No. 5,206,859 entitled ISDN MULTIMEDIA COMMUNICATIONS 

2 SYSTEM, incorporated herein by reference, describes such a 

3 system. 

4 In accordance with the invention, a variety of means can be 

5 employed to communicate selection instructions to the video 

6 server and to communicate programming material from the server to 

7 the user's display unit. Since the selection instructions are 
p8 very compact (i.e., low bandwidth), these instructions can be 

Si 9 communicated via a standard telephone link using an inexpensive, 

;;Dio low speed modem. This is similar, for example, to the method 

;l ; u cable TV viewers use to select pay-per-view programming from 

L12 local CATV companies i.e., the viewer calls a particular toll- 

!p 13 free number associated with the desired program, and the CATV 

H 14 office uses a "caller ID" function to determine which customer is 

is to receive the program. 

16 Communication of the programming material from the server to 

17 the display unit may require a higher-bandwidth means. No doubt, 

18 certain material such as pricing information for a shop-at- 

19 home service or scheduling information for a travel booking 

20 service could be easily transmitted via an inexpensive 

21 telephone/modem link. In addition, using the preferred 

22 compression/decompression techniques, many multi -media 
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1 applications could also operate over a conventional telephone 

2 link. High quality audio/video programming, however, will likely 

3 require a higher bandwidth transmission medium, such as a CATV 

4 line, microwave link, DSB link, optical fiber link, cellular 
s radio link, or enhanced bandwidth telephone connection. 

6 Although the invention has been described above with 

7 reference to several presently preferred embodiments, such 

U embodiments are merely exemplary and are not intended to define 

\|9 the scope of, or exhaustively enumerate the features of, the 

; ;Qo present invention. Accordingly, the scope of the invention shall 

;hi be defined by the following claims. Where a feature or 

limitation of a preferred embodiment is omitted in a claim, it is 

== s i3 the inventors' intent that such claim not be construed to 

qi4 impliedly require the omitted feature or limitation. 
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